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This application is related to Japanese Patent 
Application No. 2001-32960 filed on February 8, 2001, and No. 
2002-030099 filed on February 6, 2002, based on which this 
application claims priority under the Paris Convention ana 
the contents of -which are incorporated herein by reference. 

BACKGROUND OP THE INVENTION 

1. Field of the Invention 

The present invention is related to an information 
expressing method, which is suitable for TV game machines and 
the like, for example; a recording medium, which is recorded 
with an information processing program; a program execution 
apparatus; and an information processing program. 

2. Description of the Related Art 

In recent years, entertainment apparatus such as TV game 
machines, for example, have become popularized- The 
entertainment apparatus execute garre application programs 
acquired via various recording media such as CD-ROMs and 
DVD-ROMs or various communication media (transmission media) 
such as communication lines and networks and the like. 
Accordingly, users may en^oy various games. 

There are numerous types of TV games, however, among 
these, what is called a sound gams f which is a game allowing 
a game player (hereafter simply referred to as player) to 
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enjoy simulated performance Of musical instruments has gainect 
popularity . 

Among the sound games, there is a game that displays 
upon a game screen predetermined marking objects simultaneous 
to playing music, and scrolls those marking objects in a fixed 
direction such as vertical or horizontal directions 
corresponding to the tempo of the music {quickness of the 
musical piece progression) . With this sound game, the player 
may enjoy simulated musical performances by operating a 
predetermined button upon a game controller (hereafter simply 
referred to as controller) to be ON such as to coordinate wltn 
the timing of when the marking object being scrolled passes 
a predetermined point - In addition, che program of this sound 
game also carries out good/bad judgment of the player's 
rhythmic sense by measuring the matching conditions of the 
timing for a marking object to pass a predetermined point ana 
the timing for the player to manipulate a button of the 
controller to be in an ON state. 

However, in the case of the above sound game, since the 
marking object is simply scrolled in the vertical or 
horizontal direction, It is a drawback that the player has 
difficulty in grasping the tempo of the music. 

Furthermore, in the case where the tempo of the music 
has changed (for example, change from slow to fast tempo, or 
fast to slow tempo) , since the speed of scrolling the marking 
object suddenly varies, it becomes extremely difficult for 



the player to coordinate with the change in tempo. 

Moreover, it is generally difficult to judge the rhythm 
(for example, in simple triple time or quadruple time and 
such) of a piece jusz by listening to the music. Accordingly, 
in the case of conventional sound games, m order to enjoy 
the gam© (play to the rhythm of a piece) , the player is required 
to have a strong sense of rhythm, making it difficult for a 
player without confidence in rhythmic sense to play the game. 

As in the above, the conventional sound games make it 
difficult for the player to grasp the tempo of the music, ana 
are of high difficulty, requiring a strong sense of rhythm, 

SUMMARY OF THE INVENT I OK 

The present invention takes such problems into 
consideration, and aims to make it easy to grasp the timing 
of predetermined information and to easily determine the 
cycle of the information. Namely, the present invention aims 
to provide an information expressing method; a recording 
medium, wnich is recorded with an information processing , 
program; a program execution apparatus; and an information 
processing program, allowing the player to easily grasp the 
tempo of the music, easily determine the rhythm of the piece, 
and further enables hlm/ner to carry out a musical performance, 
for example, as an orchestral conductor. 

The present invention expresses by the number of the 
predetermined objects a structure of repeating periodically 



changing information; and expresses the progressing speed of 
the information by the distance among the predetermined 
objects and a moving object that moves among the predetermined 
ob] ect s , 

The structure of repeating periodically changing 
information may be a structure of repeating periodically, for 
example, strong and weak sounds, such as the rhythm of the 
music. The progressing speed of tne information may be the 
progressing speed of, for example, a musical piece such as 
the tempo of the music. Namely, the present invention 
expresses the rhythm of the music by, for example, the number 
of the predetermined objects, and expresses the tempo of the 
music by the distance among the predetermined objects and the 
moving of the moving coject. 

Artisan will fully understand the above and further 
objects and features of the present invention by reading 
preferred embodiments of the invention described herein in 
connection with the accompanying drawings, including 
features recited in the appended claims, as well as explicit 
and implicit effects ana advantages of the invention, 
including those to be apparent through implementation of the 
invention . 

BRIEF DESCRIPTION OF DRAWINGS 

FIG. 1 is a schematic diagram for the description of 
the arrangement of timing marks in the case of simple triple 



time and a riming ball moving between each riming mark. 

FIG. 2 is a schematic diagram for the description of 
expressing tempo by the distance among the timing marks* 

FIG. 3 is a schematic diagram for the description of 
the arrangement of timing marks in the case of quadruple time 
and a timing ball moving between each timing mark. 

FIG. 4 is a schematic diagram for the description of 
the beat. 

FIG. 5 is a schematic diagram for the description of 
a depicted circle mark in the case of succeeding in the timing 
on the beat. 

FIG. 6 is a schematic diagram for the description of 
a depicted X mark in the case of failing m the timing on the 
neat and a depicted afterimage m the case where the timing 
on the beat is late. 

FIG. 7 is a schematic diagram for the description of 
a depicted X mark in the case of failing to hit the beat and 
a depicted afterimage in rhe case where the timing on the beat 
is early. 

fig. 8 is a schematic diagram showing the condition of 
volume expressed by color of the timing marks. 

FIG. 3 is a schematic diagram showing the condition of 
a parr designation expressed by a pare designating arrow icon 
upon a timing mark. 

FIG. 10 is a schematic diagram showing the condition 
of sostenuto expressed r>y a star mark upon a timing mark. 



FIG. 11 is a schematic diagram showing a displaying 
example Of right/wrong judgment results for every measure of 
game playing. 

FIG. 12 is a schematic diagram showing an example of 
a TV game display for the sound game of this embodiment. 

FIG . 13 is a configuration of data for the sound game 
application program of this embodiment. 

FIG. 14 is a flowchart showing the entire process 
associated with simulated conducting and producing of music 
for the sound game application program of this embodiment - 

FIG. 15 is a flowchart showing a specific example of 
processing timing mark: display and timing ball movement. 

FIG . 16 is a flowchart showing an example of processing 
OK/NG judgment of the timing on the beat, OK/NG judgment of 
pushing of buttons and the like according to marJc color and 

symbol, pressure-sensed value measurement and OK/NG judgment 
and the liice. 

FIG. 17 is a flowchart showing an example of processing 
display and sound output according to OK/NG judgment results 
of the timing on the Oeat. 

FIG . 18 is a flowchart showing an example of processing 
display and sound output according to OK/NG judgment results 
of th« pressure-sensed value. 

FIG . 19 is a flowchart showing an example of processing 
display and sound output according to part indication 
judgment results. 



FIG. 20 a.3 a flowchart snowing an example of processing 
display ana sound output according to sostenuto judgment 
results. 

FIG * 21 is a schematic diagram snowing an example of 
the scnemailC structure of the main sections of the TV game 
system of the embodiment of the present invention. 

FIG. 22 is a block diagram showing the internal schematic 
structure of the TV game machine of this embodiment, 

FIG . 23 is a cross-sectional view cut along a line in 
the pressure direction of a pressure-sensitive button in a 
non-pushed condition. 

FIG. 24 is a cros s - sect lonal view cut along a line m 
the pressure direction of a pressure-sensitive button in a 
partly pushed condition. 

PIG. 25 is a cross-sectional view cut along a line in 
the pressure direction of a pressure-sensitive button in a 
fully pressed condition. 

FIG. 26 is a circuit diagram showing the schematic 
structure of pressure-sensitive resistor with a resistance 
value that varies according to pushing pressure. 

FIG - 27 is a diagram showing variation in pushing 
pressure (horizontal axis) applied to a pressure-sensitive 
button and voltage values (vertical axis) of electric signals 
corresponding to pushing pressure. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Various embodiments of the present invention will be 



described with reference to the accompanying drawings. It 
is to be noted that the same or similar reference numerals 
are applied to tne same or similar parrs and elements 
throughout the drawings, and the description of the same or 
similar parts and elements will be omitted or simplified. 

Brief Description of TV Game of the Embodiment 

An embodiment of a TV game that the present invention 
is applied to is a sound game allowing a player to simulatively 
conduct a musical performance as, for example, an orcnestral 
conductor, and carry out the management of musical pieces and 
produce music; in particular, it is one allowing tne player 
to play the game while visually confirming tne tempo and 
rhythm of the music. 

The game program of this sound game displays upon a 
television monitor screen (TV game screen) timing marks TM 
and timing ball TB as described below with FIG. 1 through FIG * 
12. Then, the game program expresses each element that is 
needed when simulatively conducting or producing music such 
as tempo, rhythm,, beat, volume, part designations, and 
sostenuto and the like based on the distance among timing 
marks TM (length between marks, distance of empty space), 
number of the marks, tne display coj or of each mark, symbol 
displayed upon the marks, the moving of timing ball TM and 
the like. In this manner, the game program visually informs 
the player of each element required when simulatively 



conducting or producing music based on the distance among 
timing marks TM, number of marks, the display color of eacn 
mark, the symbol displayed upon the marks, the moving of 
timing ball TM and the like- 
Moreover/ the game program detects that a button upon a 
controller is operated co be ON by the player so as to adjust 
Che length of sostenaro. In addition, the game program 
detects the pushing pressure on the button so as to adjust 
the volume according to that pressure-sensed value. 
Farthermore, the game program detects the operation of 
direction keys upon the controller by the player. 
Accordingly, the game program recognizes the command input 
for simulated conducting and producing by the player. 

It is noted that details of the TV game system of this 
embodiment, the controller, the buttons upon that controller, 
the means for detecting pushing pressure when operating those 
buttons to be ON, and direction keys upon the controller and 
the like will be described later. 

Details of Timing Marks and Timing Sail and Expression 
Examples of Each Element 

FIG. 1 through FIG . 12 show examples of timing mark TM 
arrangement and specific examples of timing ball TB moving 
among timing marks TM. 

The game program displays timing marks TM in 
predetermined positions (for example, easily seen position 



such as center screen) upon the TV game screen. Furthermore, 
the game program displays timing ball TB so as to mova among 
riming marks TM at a fixed speed. This game program visually 
informs the player in an easy to understand fashion of the 
following six types of elements (elements necessary for 
simulated conducting and producing of music) such as tempo, 
rhythm, beat, volume, part designations and sostenuto by 
showing such as the arrangement of timing marks TM, number 
of the marks, distance among marks, trie mark color, the symbol 
displayed upon the mar*s, ana by the moving of a timing ball 
TB . 

Expression of Tempo 

This sound game expresses the tempo of pieces {quickness 
of musical piece progression) by the distance among timing 
marks TM, Namely, this sound game expresses the tempo of a 
piece by the distance among timing marks TM (length among 
marks) under the condition of moving timing ball TB at a fixed 
speed between each timing mark TM. More specifically, the 
game program coordinates the ideal tempo of the musical piece 
with the intervals of time between each timing mark TM which 
are decided by timing ball TB moving at a fixed spe»d and 
overlapping with each timing mark TM in turn. Namely, this 
game program, for example as shown in FIG. 1, informs the 
player that the tempo of the musical piece is slow by 
lengthening the distance among timing marks TM so as to 
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lengthen the intervals o£ rime required for moving rhe riming 
ball TB between each timing mark TM. Furthermore, rhe game 
program, as shown in FIG. 2, informs rhe player that the tempo 
of rhe musical piece is fasr by shortening the distance among 
timing marks TM so as ro shorren rhe intervals of time required 
for moving the timing ball TB among those timing marks TM. 
In this manner, the game program, by the length of the distance 
among timing marks TM, visually informs the player in an easy 
to understand fashion tne tempo of rne musical piece- 
Furthermore, the game program, according to need, 
changes rhe distance between each riming mark TM even 
mid-musical piece. Namely, wnen rhe tempo is altered (tempo 
change) mid-musical piece, the game program changes the 
distance between each riming mark TM according to that 
alteration. In this manner, by changing rhe distance among 
riming marics TM, the game program visually informs the player 
in an easy to understand fashion whether or not there has been 
a tempo change and what the tempo is "thereafter. Moreover, 
rhe displaying position of each timing mark TM is shifted at 
the point when riming call TB passes, for example, timing mark 
TM that corresponds to one measure before rhe riming where 
tempo change of the music is actually carried out, and changes 
the distance between each OZ rnose ciming marks TM to the 
distance corresponding to the tempo thereafter. In this 
manner, by tne move in displaying position of each timing mark 
TM, rhe game program visually informs the player m an easy 
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to understand fashion that there is a tempo change mid-piece. 
Furthermore, by changing the distance between each timing 
mark TM, the game program visually informs the player in 
advance (one measure before tempo change is actually carried 
out) that tempo change will occur and the tempo thereafter. 

It is noted that when tempo change occurs, the game 
program alters the motion of characters and background images 
upon the TV game screen accordingly. For example, when the 
tempo is fixed, the background and characters move in 
accordance with that tempo; however, when that tempo changes, 
the game program alters movement of the characters ana actual 
background and images. 

Expression of Rhythm 

This sound game expresses the rhythm of musical pieces 
(structure of music wnerem the strong/weak sounds are 
periodically repeated) by the number of timing marks TM . In 
other words, the game program, for example as in FIG, 1 and 
FIG- 2 r by setting the number of timing marks to be 3 marks, 
informs the player that it is a piece in simple triple time 
and as in FIG. 3, by setting the number of marks to be 4 marks, 
informs the player that it is a piece in quadruple time. It 
is noted that FIG. 1 and FIG. 2 are just an example, and by 
setting the number of marks to be 5 marks, the game program 
visually informs the player that it is a piece in quintuple 
time. In this manner, the game program visually informs the 
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player m an easy to understand fashion the rhythm of the 
musical piece by the number of timing marks TM- 

Furthermore, the game program essentially corresponds 
one measure of a musical piece With the cycle, wherein timing 
ball TB shifts in turn between each of three or four timing 
marks TM coming full circle. For example, in the case of 
triple time as shown in fig. i and FIG . 2, the game program 
corresponds one measure of a musical piecg in triple time with 
one cycle of timing call TB moving m the order of timing marie 
TM1 — * TM2 — » TM3 -» TM1 ♦ Furthermore, m the case of 
quadruple time shown in FIG. 3, for example, the game program 
corresponds one measure of a musical piece in quadruple time 
with one cycle of timing ball TB moving in the order of timing 
mark TM1 — » TM2 — » TM3 — * TM4 — ► TM1 . In this manner, the 
game program, by corresponding one measure of the musical 
piece with one cycle that timing ball TB shifts, visually 
informs the player in an easy to understand fashion one 
measure of a musical piece. 

Moreover, according to need, the game program changes 
the number of the timing marks TM, even mid-musical piece. 
Namely, in the case where Che rhythm is altered (change in 
rhythm) mid-musical piece, the game program changes the 
number of the timing marks TM according to that alteration. 
In tnis manner, the game program visually informs the player 
the occurrence of change in rhythm and the rhythm thereafter. 
Furthermore, the game program changes the number of timing 
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marks TM one measure before the riming of altering the rhythm 
to the number cf marks corresponding to the rhythm thereafter. 
In this manner, by changing the number of timing marks TM, 
the game program visually informs the player in an easy to 
understand fashion that the rhythm will be altered mid-piece; 
moreover, informs the player that variable rhythm will occur 
on© measure before the rhythm is actually altered. 

Expression of Timing on the Beat 

Here, the timing on the seat of the musical piece will 
be referred to as the beat. The game program depicts a symbol 
corresponding to a predetermined button of the controller 
upon a deslrect timing mark TM. * In this manner, by depicting 
upon the desired timing mark TM a symbol corresponding to the 
predetermined button, the game program visually informs the 
player of the manipulation to be carried out at the timing 
on the beat* 

More specifically, as shown in FIG. 4, the game program 
depicts upon the desired timing mark TJM2 a symbol 
corresponding to the predetermined button (for example, 
button described later) of the controller, requesting the 
player to perform pushing manipulation of the button in 

accordance with overlapping of timing ball TB and timing mark 
TM2. Then, according to that request, when the pushing 
manipulation of the -A" button is performea by the player, 
the game program proceeds with the musical piece one oeat by 



14 



hitting on the beat, thus raising the game points. 

Here, as shown in PIG. S, when, the pushing manipulation 
of the X *A" button is carried out by the? player and the beat 
is hit precisely in accordance wich overlapping of timing ball 
T.B and the desired timing mark TM2 , the game program depicts 
upon that timing mark TM2 circle mark Gm, which indicates that 
the timing of the pushing manipulation of the V A" button by 
the player is accurate. On the other hand, as shown in FIG. 
6 and FIG. 7, in the case where the pushing manipulation of 
the button is carried out by the player and the beat is 

not hit precisely when timing ball TM and the desired timing 
mark TM2 do not overlap, the game program depicts upon that 
timing mark TM2 X mark Brn, which indicates that the timing 
of the pushing manipulation of the %X A" button by the player 
is not accurate. 

Furthermore, m the case where the "A" button is pressed 
by the player after timing ball TB passes the desired timing 
mark TM2, in other words, when the timing of pushing the ''A" 
button is late, the game program, as shown in FIG • G, depicts 
afterimage AB of timing ball TB, which passed over timing mark 
TM2 . on the contrary, m the case where the "A" button is 
pressed by the player before timing ball TB reaches the 
desired riming mark TM2 , in other words, when the riming of 
pushing the "A" button is early, the game program, as shown 
in FIG. 7 , depicts afterimage AB of timing ball TB in a position 
before it reaches timing mark TM2 . It is noted that 
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afterimage AB of wnen the riming of button pushing is late 
may be depicted in a position before it reaches timing mark 
TM2 ; and afterimage AB of when the timing of button pushing 
is early may be depicted in a position it passed over timing 

mark TM2 . 

As in the above, zhe game program informs the player 
in an easy to understand fashion whether or not the beat is 
precisely hit, and in the case where the beat is not precisely 
hie, wnether the timing on that beat is too early or too late. 

Expression of Volume 

me TV game system is made capable of detecting the 
pushing pressure on the button upon the controller. 
Furtnermore, the game program is capable of setting the 
magnitude of volume of the music (strong/weak musical 
performance) according to the pushing pressure of that 
button . 

Furthermore, this sound game expresses the magnitude 
of volume (strong/weak performance) required for when the 
player simulatively conducts or produces by using the display 
color, which is a parameter when displaying upon the monitor 
screen timing marks TM . More specifically, by setting the 
display color of timing mark TM to be v *red", the game program 
informs the player that he/she is prompted for an instruction 
for performing with high volume (perform "strong"); by 
setting the display color of timing mark TN. to be M green", 
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informs the player thai he/sfie IS prompted for an instruction 
for performing with moderate volume (perform "medium") ; and 
by setting the display color of timing mark TM to be "BLUE", 
informs the player that he/she is prompted for an instruction 
for performing with low volume (perform "weak") . 

For example, in the case where the player is prompted 
to adjust the pushing pressure on the button upon the 
controller such that low volume (perform x, weak") — ► high 
volume (perform "strong") — » moderate volume (perform 
"medium") — > low volume (perform "weak") , the game program, 
as shown in FIG . 8, sets the display color of timing mark TM1 
to be "blue"; sees the display color of timing mark TM2 to 
be "red"; and sets the display color of timing mark TM3 to 
be "green"; moreover, moves riming ball TB in the order of 
timing mark TMl —> TM2 — > TM3 — > TM1 - 

In this manner, by the display color of each timing mark 
TM, the game program informs the player of the magnitude of 
volume (strong/weak performance) when performing music. 

Expression of Part Designations 

By depicting upon a desired timing mark TM an arrow 
symbol corresponding to a desired direction key of the 
controller, the game program visually informs the player that 
he/sne is prompted for a designation of a part, performer, 
instrument and the like of a virtual orchestra or band; and 
an instruction for carrying out the performance of that part, 
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performer, instrument and the like. 

More specifically, 33 shown m FIG. 9, by depicting upon 
the desired timing mark: TM2 part designating arrow icon Pm 
for designating a desired part, performer or instrument, ana 
by making a relationship between part designating arrow icon 
Pm and a desired timing mark TM2, the game program requests 
that the player should perform the pushing manipulation of 
a direction key ad the same time that timing ball TB is 
overlapped witn timing mark TM2, m which part designating 
arrow icon Pm is depicted. The game program, according to 
that request, has carried oat the performance of the desired 
part, performer, instrument and the like when the pushing 
manipulation of the direction key is carried out by tne player 
On tne other hand, in the case where the pushing manipulation 
of the direction Key, is not carried out at the same time that 
timing mark TM2 is overlapped with timing ball TB, and in the 
case where the direction key that differs from the direction 
designated by part designating arrow icon Pm is pushed, the 
game program does not carry out the performance of the part r 
performer, and instrument. 

It is noted that this sound game is a game assuming the 
condition where the part, performer and instrument are 
arranged in front of the conductor or on both sides thereof 
as in an orchestra. As a result, the game program displays 
part designating arrow icon Pm as a symbol that designates 
a direction from three directions corresponding ro the three 
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direction keys of M up", M left" and ^right" described later 
upon the controller. 

Expression of Sostenut o 
5 When the button upon the controller is continuously 

pushed, the game program stops the progression of music by 
prolonging the sound or silent state when that button IS in 
an ON state. Then, by depicting the predetermined symbol upon 
the desired timing mark TM, the game program visually informs 
10 the player that he/she is prompted for an instruction 
in£ regarding the act of intentionally prolonging tne length of 
pi sound at tne time of a musical performance, 

pi More specifically, the game program depicts upon tne 

^ desired timing mark TM2 star mark icon Sm shown in FIG. 10 

Cn IS as a symbol for prompting an instruction for prolonging the 
0 length of sound or silent state. Then, when the button upon 

Q the controller is continuously pushed by the player for a 
^ predetermined set: duration in accordance with overlapoing of 

iy timing ball TB and timing mark TM2 , which has depicted star 
20 mark Sm, tne game program halts the progression of music by 
prolonging the sound or silent state at that time. 

Furthermore, the game program sets the continuous 
Pushing duration of the outton for stopping the progression 
of music to the duration corresponding to tne time it takes 
25 timing ball TB to move to the next timing mark TM. Furthermore, 
with the game program, the player's ON re-mampjlacion of the 
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same button on the controller satisfies the cancellation 
conditions (resuming conditions for progression of music) for 
halting progression of the music* It ;s noted that the reason 
for the cancellation conditions of the 

progressing/suspending status of the music as described above 
is satisfied with the player's ON re-manipulation of the same 
burton upon the controller is because, for example, if release 
of the continuously pressed button is set as the cancellation 
condition for the suspending status, a commanding 
manipulation, which is necessary for the player to enter the 
next stage so as to give an instruction for the next tempo, 
is difficult for the player. Moreover, when progression o£ 
the music has been halted, the game program orbits timing ball 
TB or the afterimage thereof, for example, above the periphery 
of timing mark TM. By displaying such image, the game program 
visually informs the player that progression of the music has 
been halted. 

In this manner, the game program visually informs the 
player that he/she is prompted for an instruction for 
intentionally prolonging the length of sound or silent state. 

By continuously pressing the button upon the controller 
so as to halt progression of the music Joy prolonging the sound 
or silent state while that button is in an ON state, the player 
carries out what is called a performance that "holds" notes 
or a performance that lengthens what is called "a rest". 

As described above, by timing marks TM and timing ball 
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TB, this sound game can inform the player each element, such 
as tempo, rhythm, beat, volume, part designations, sosienuto 
and the like, that are necessary for simulatively conducting 
music or carrying out a musical production and allowing the 
player to carry out simulated conducting and producing of 
music . 

Furthermore, this sound game is prepared with various 
other modes as well as the normal performance mode as 
described aJoove . In normal performance mode, the game 
program prompts the player for an instruction for a 
strong/weak performance by the display color of timing marks 
TM; prompts tne player for a designation of a part by part 
designating arrow icon Pm depicted upon timing mark TM; and 
prompts the player for an instruction for sostenuto by star 
mark Situ In ocher moaes, the game program, for example, makes 
the player carry out estimation according to musical content 
and prompts the player for an instruction for a strong/weak 
performance; a designation for the to-be-performed part; and 
an instruction for the degree of sostenuto at the player' s 
volition. In the mode where the player personally carries 
out an instruction for a strong/weak performance, a 
designation of to-be-performed part, or an instruction for 
tne degree of sostenuto, the game program sets the display 
color for eacn of all timing marjes TM to be "yellow", and does 
not display part designating arrow icon Pm or star mark Sm. 
Namely, when the display color for each of all timing marks 
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TM is made to be "yellow", the player must personally carry 
out: an instruction for a strong/weak performance, a 
designation of a part, and an instruction for sostenuto 
according to musical content . when each of those 
instructions matches the musical content, the game program 
raises the game points - It is noted that the game program 
sets the mode for the player to personally carry out various 
instructions, for example, in the mode of continuously 
performing a plurality of musical pieces (medley mode). 
Furthermore, in normal performance mode, the game program, 
for example, displays upon timing marks TM the marks of 
buttons such as "Zl", "D", "X", and "O", requests the player 
to push those nuttons and shifts from the normal performance 
mode into the medley mode when those buttons are pushed. 

Expression of Game Completion 

The game program judges whether or not the player has 
fulfilled the request concerning the tempo, rhythm, beat, 
volume, part designations, sostenuto and the like while 
timing ball TB moves m turn between each of the timing marks 
come full circle (for every measure) . In the case where the 
game program judges that those requests have been fulfilled, 
for example as shown m FIG. 11, the game program carries out 
the display of such characters as ^Good!" or "Bravo!" upon 
a predetermined timing mark TM (for example, TM1 ) , as well 
as carrying out sound output such as "Good" or "Bravo". 
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Accordingly, the player understands that a game one measure 
long has successfully been completed. On trie contrary, m 
the case where the player has not fulfilled the requests while 
timing ball TB makes full circle of each of the timing marks, 
tne game program, for example, displays upon timing mark TMl 
the characters "Bad!" or a skull mark or the like, as well 
as carrying out sound output of "Bad" and the like. 
Accordingly, the player understands that a game one measure 
long has not been completed. 

The game program, as Shown in FIG . 12, displays upon 
TV game screen 250 timing marks TM ( TMl to TMfl in the example 
of FIG. 12), timing ball TB and background image, as well as 
tension meter 251 and pressure-sensing meter 252. 

Tension meter 251 comprises color-coded meter levels 
and game completion standard line CL. The game program raises 
and lowers the meter level according to the increasing and 
decreasing of game points. Then, the game program informs 
the player of the degree of completion of the game by the meter 
level value of tension meter 251 and color thereof. It is 
noted that the game program proceeds with the game to the next 
stage (scene) when the meter level according to the game 
points exceeds standard line CL. 

Pressure-sensing meter 252 comprises color-coded meter 
levels. The game program raises and lowers the meter level 
according to the pushing pressure (pressure-sensed value) on 
buttons when expressing volume. Then, the game program 
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informs the player of the pashmg pressure intensity on the 
buttons by the meter level value of pressure-sensing meter 
252 and color thereof. It is noted that the color of the meter 
level value of pressure-sensing merer 252 corresponds to the 
color-coding of timing marks TM . 

The game program also alters the background images 
(design and color and the like) in accordance with the rise 
ana fall of the meter level (increase and decrease of game 
points) of tension meter 251. 

Schematic Structure of Game Application Program 

Next, the structure of a game program for providing the 
above-mentioned s o and game of this embodiment will be 
described . 

The game program has a data structure as shown in FIG. 
13, for example, whereby it may be recorded on recording media 
such as semiconductor memory and optical disks, for example, 
DVD-ROMs and CD-ROMs, and may be downloaded via transmission 
media such as communication lines. It is noted that this data 
structure shown in FIG . 13, as described above, is one that 
conceptually represents only the main sections of a program 
section and data section included in this game program 
allowing the player no carry out simulated conducting or 
producing of music, and does not represent an actual program 
structure . 

As shown in FIG* 13, roughly classified, game program 
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330 has program section 340 for a main CPU and ihe like of 
a TV game machine described later to execute a sound game, 
and data section 360 including various data, which is to be 
utilized when executing the sound game. 

Data section 360 has at least polygon/texture data and 
such 361 and sound source data 362 as various data tnat are 
utilized when executing the sound game. 

Polygon/texture data and such 361 is data for generation 
of polygon or texture when generating game images that include 
characters such as the main character or performer or the like, 
and oackgrouna images and the like within the game* Sound 
source data 362 is waveform data, which is utilized in 
generating sound game sounds, music of musical pieces, sound 
effects and the like by a sound processing umr of a TV game 
machine described later. 

Program section 340 has at least progression control 
program 341, disk control program 342, controller management 
program 34 3 , image control program 34 4 , sound control program 
345, musical piece management program 346, timing mark 
control program 347, ball control program 348, character 
control program 349, meter management program 350 and saving 
management program 351 as programs for executing the sound 
game - 

Progression control program 341 controls progression 
of the sound game. Disk control program 3d2 controls reading 
of data and the like from the optical disks and HDDs according 
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"eo the beginning or progression of the sound game. Controller 
management program 343 manages the input signals m response 
to the player's pushing manipulation of buttons upon a game 
controller, tne operating moae of the controller, vibration 
generation described later and the like. Image control 
program 344 generates game images and displays the game images 
upon a television monitor screen. Sound control program 345 
generates and outputs sound and music within the sound game. 
Musical piece management program 346 manages each of the 
elements such as performing order of musical pieces performed 
by the sound game; the start, progress and halt of musical 
pieces; tempo; rhythm; beat; volume; part designations; 
sostenuto and the luce. Timing mark control program 347 
carries out the control of arranged position, number of marks, 
color, symbol and the like of timing marks TM . Ball control 
program 348 controls the movement of timing ball TB. 
Character control program 349 controls the movements and 
actions of characters appearing within the sound game- Meter 
management program 350 manages the meter levels of tension 
meter 251 and pressure-sensing meter 252, which are displayed 
upon the monitor screen. Saving management program 351 
manages saved data, such as by storing the game points 
generated in the souna game and mid-game data and the like 
as the saved data on a memory card described later, and reading 
and tne like the saved data that is stored on the memory card. 
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Flew of Gamg Application Curing Execution 

In the following, processing flow of the game program 
snown in FIG. 13 described above and the flow of displaying 
and moving timing marks TM and timing ball T3 displayed upon 
5 the monitor screen by the sound game will be described wnile 
referencing the flowcharts of FIG. 14 through FIG . 20. It 
is noted that the steps of each flowchart described below are 
provided running each program component, which configures the 
game program, with the built-in CPU of the TV game machine 
10 described later. Furthermore, in the description below, 
H : particularly only the components relevant to the processing 

Q characteristic of the present invention, which allows the 

M 

fy player to carry oat Simulated conducting and producing of 

s& s 

If; music, are extracted and mentioned. 

Overal l Processing Flow of Sound Game 

Q FIG. 14 shows an example of the overall flow of 

m 

0 processing of game program 330 that allows the player to carry 
out tne simulated conducting and producing of music. 
2U To begin with, for example, when the switch of the TV 

game machine described later is turned ON and execution of 
the sound game commences by loading, for example, an optical 
aisk and the like, during the processing of step SI, 
progression control program 341 displays upon a television 
25 monitor screen a game initial screen; moreover, during the 
processing of step S2, displays a menu screen and the like, 
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which requests the player to select: a musical piece to be 
performed in the sound game. It is noted that with the sound 
game of the present invention, the game program may allow for 
selection of musical pieces not only by the player, bat also 
by automatic selection in accordance with progression of the 
game. When selection of a musical piece to be performed is 
carried oat in Step S2, sound control program 345 and musical 
piece management program 346 commence tile performance of the 
musical piece during the processing of step S3. 

Simultaneous to commencing the performance of the 
musical piece, during the processing of step 54, mark control 
program 347 and ball control program 348 display timing marks 
TM tnat are expressed by the distance among marks, number of 
marks, display color of each mark, symbol upon che mark and 
the like, according to the six types of elements of musical 
tempo, rhythm, beat, volume, part designations and sostcnuto, 
which are acquired from musical piece management program 346; 
moreover/ carries out the moving of timing ball TB. 

While displaying of timing marks TM and moving of timing 
ball TB are being carried out, during the processing Of Step 
55, based on the information such as the displaying condition 
of timing marks TM and the operating signals from the 
controller, and the movement conditions of timing ball TB and 
the like, progression control program 341 carries out the 
OK/ NG judgment of timing on the beat; the OK/NG judgment of 
pusning buttons or keys according to color of the mark and 



the symbol upon the mark; measurement: of the pressure- sensed 
value corresponding to the pushing pressure on the button and 
the OK/NG judgment thereof and the like while working together 
with controller management program 343, mark control program 

347 and ball control program 348. 

while displaying of timing marks TM and moving of timing 
ball TB arc being carried out, during the processing of step 
S6 f based on the results of the OK/NG judgment of timing on 
trie beat; the OK/NG judgment Of pushing buttons or Keys based 
on symbol and the like upon the mark; and the measurement of 
the pressure-sensed value corresponding to the color of the 
marks, progression control program 341 carries out display 
and sound output of "Good!" or "Bravo I" or the like carried 
out for every measure , level display of the pressure-sensing 
meter 252, addition/subtraction of game points and tne like 
while working together with image control program 344, sound 
control program 345, character control program 349 and meter 
management program 350. 

Next, during the processing of step S7 , meter management 
program 350 carries out level setting and display of tension 
meter 251 according to tne game points obtained in step S6. 

Next, during the processing of step S8, progression 
control program 341 judges whether or nor rne musical piece 
has ended, returns to the processing of step 54 wnen it has 
not ended, and when the end of the musical piece is judged, 
accumulates the game points during the processing of step S9. 
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Afterwards, during tine processing of step S10, 
progression control program 341 displays upon the monitor 
screen the game results according to the accumulated game 
points and ends the sound game. 

Hereafter, the main steps from each of the processes 
of the flowchart of FIG. 14 will be described. 

Process Flow of Timing MarK Display and Timing Ball Movement 
FIG. 15 shows a specific example of the process flow 
of timing marie display and timing bail movement in step S4 
of the flowcnart of FIG. 14. 

In FIG. 15, when the musical piece performance commences 
in step S3 of FIG. 14, marie control program 34"7 and ball control 
program 348 , to begin with, during the processing of step S22, 
carry out the initial settings of the number of timing marks 
TM, distance among the marks, mark color and the like 
according to the rhythm, tempo, volume and tne like of the 
musical piece that musical piece management program 346 
manages; moreover, during tne processing of step S23, 
commences moving the timing £>all. 

Next, mark control program 347 and ball control program 
343, based on the six types of elements of musical tempo, 
rhythm, beat, volume, part designations and sostenuto that 
musical piece management program 346 manages, judge during 
the processing o£ step 324 whether or not setting of the beat 
has been carried out for each rhythm of each measure; during 



the processing of step S26 whether or not setting of volume 
change has been carried oat; during the processing of step 
S28 whether or not setting of the part designation has been 
carried out; during the processing of step S30 whether or not 
setting of sostenato has been carried out; during the 
processing of step S32 whether or not setting of the tempo 
change has been carried out; and daring the processing of step 
S34 whetner or not setting of change of rhythm has been carried 
out. It is noted that trie order that each judgment of these 
steps S24, S2G, S28, S30, S32 and S34 is carried out is not 
limited to the example of FIG. 15. 

Namely, when setting of the beat for eacn rhythm is 
carried out m step S24, during the processing of step S25, 
mark: control program 347 depicts a symbol, which represents, 

for example, the "A" but con upon the controller as the symbol 
prompting the player for timing on tfi* beat, upon timing mark 
TM, which represents the beat. 

When change setting of the volume for each beat 
(intensity of performance) is carried out in step S26, during 
tne processing of step S27, mark control program 347 sets the 
color of timing marie TM that represents that beat to the color 
according to the specified value of the volanie. 

When setting of the part designation for each beat is 
carried oat in step S28, during the processing of step 529, 
mark control program 347 depicts part designating arrow icon 
Pm, which corresponds to a direction key upon the controller, 
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for example, as the symbol prompting me player for tne parr 
designation, upon timing mark TM, which represents that beat. 

When setting of sosienuto for eacn beat is carried out 
m step S30, during the processing of step 531, mark: control 
program 347 depicts a star mark Sm, for example, as the symbol 
prompting the player for sostenuto, upon timing mark TM that 
represents that beat. 

When setting of tempo change for each measure is carried 
out m step S32, after timing ball TS passes timing mark TM 
that corresponds to, for example, one measure before the poinc 
where tempo change occurs, mark control program 347, during 
the processing of step S33, changes the distance between each 
timing mark TM in accordance with the tempo thereafter. 

When setting of rhythmic change for each, measure is 
carried out in step S34, after timing ball TE passes timing 
mark TM that corresponds to, for example, one measure before 
the point where the rhythm is altered, mark control program 
347, curing the processing of step S35, changes the number 
of the timing marks to the number of the marks in accordance 
with that altered rhythm. 

Process Flow of Button Pushing OK/NG Judgment, 
P ressure-sensed Value Measurement and Judgment 

In PIG. 16, an example of tne process flow of the OK/NG 
judgment of timing on the beat; the OK/NG juagment of pushing 
a button or key in response to color of the mark or the symbol 
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upon the mark; the measurement and OK/NG judgment of trie 
intensity of the pushing pressure on the button 
(pressure-sensed value) and the like are shown in step 35 of 
the flowchart of FIG . 14. It is noted that each nudging 
process shown in FIG. 16 is actually carried out 
simultaneously. 

in FIG. 16, progression control program 341, based on 
the color and symbol upon timing mark TM that mark control 
program 347 controls; the moving of timing ball TB that ball 
control program 348 controls; and the ON operating signal for 
the button or key upon the controller that controller 
management program 343 manages, to begin with, during tne 
processing of step 541, carries our judgment o£ whether or 
not the desired button upon the controller is manipulated to 
be ON at the point when timing ball TB overlaps upon the cimlng 
mark (with the timing on the beat). 

When the desired button is manipulated to be ON with 
the timing on the beat in step S41, progression control 
program 341, during the processing of step S42, carries out 
an OK judgment for the timing on the beat. On the other hana, 
in step 341, when the desired button is not manipulated to 
be ON with the timing on the beat, progression control program 
341, during the processing of step S43, carries out an NG 
judgment for the timing on the beat. 

Next, progression control program 341, based on the mark 
color of tne timing marks tm (namely volume size) that mark 
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control program 347 controls; the moving of riming ball T B 
that ball control program 348 controls; and the ON operating 
signal for the button or key upon the controller that 
controller management program 343 manages, during the 
processing of step S44, carries out judgment for whether or 
not the pressure sensitive value, which represents the 
magnitude of the pushing pressure on the desired button upon 
the controller, is set at the value corresponding to the 
volume set for each beat (for each timing mark) * 

when it is judged in step S44 that the pres sure - sensed 
value of the pushing pressure on the button is the value 
corresponding to the set volume, progression control program 
341 carries out a volume OK judgment during the processing 
of step S45. On the other hand, when it is judged tnat the 
pressure-sensed value of the pushing pressure on the button 
is not the value corresponding to the set volume, progression 
control program 341 carries out: a volume NG judgment during 
the processing of step S46. 

It is noted that controller management program 343 
measures at every 1/60 TA second the magnitude of the pushing 
pressure on the button upon the controller as the 
pressure-sensed value expressed in 256 levels; moreover, 
based on four consecutive pressure-sensed values tnat are 
acquired at every l/60 rn second/ carries out a pushing 
pressure judgment for during the button ON manipulation by 
the player. More specifically, when Che player is requested 
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to carry our me ON manipulation of 'a button for high volume 
(perform "strong") f controller management program 343 
extracts as the value of tne pushing pressure on the button 
by tne player tne maximum pressure-sensed value from the four 
consecutive pressure-sensed values acquired at every 1/60 CJi 
second when the burton is manipulated to be ON, On the other 
hand, when the player is requested to carry out the ON 
manipulation of a button for lew volume (perform %% weak"), 
controller management program 343 extracts as the value of 
the pushing pressure on the button by the player the minimum 

pressure-sensed value from the four consecutive 
pressure-sensed values acquired at every 1/60" second when 
the button is manipulated to be ON. Moreover, when the player 
is requested to carry out the ON manipulation of a button for 
medium volume (perform "riddle") , controller management 
program 343 extracts as tne value of the pushing pressure on 
the button by the player tne pressure-sensed value almost in 
the middle from the four consecutive pressure-sensed values 
acquired at every l/SQ zh second when the button is manipulated 
to be ON. . Judgment of the pushing pressure is carried out 
based on the four consecutive pressure-sensed values as 
described above because as the button ON manipulation by man 
is not strongest or weakest the instant the button is pushed, 
whereby if the pushing pressure on the button is only ]udged 
at the instant the button is pushed, the probability of the 
player's intended button pushing pressure that is not being 
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judged is nigh. Furthermore, the game program is capable of 
setting the intensity of the pushing pressure on the button 
using the initial screen and the like. In other words, the 
player can preset the pushing force of the button that he/she 
5 pushed, and the degree of intensity thereof that the game 
program will judge. 

Next, progression control program 341, based on the 
symbol (in this case, the direction that the arrow mark of 
part designating arrow icon Pm points) upon timing mark TM, 
10 which marjc control program 347 controls; tne move of timing 
h& ball TS, which ball control program 348 controls; and the 
55 operating signal of the direction keys upon the controller, 

H wnich controller management program 343 manages, during the 
I'll processing of step S47, carries out judgment of whether or 
||| is net the direction key corresponding to the direction part 
p instruction arrow icon Pm is pointing has been pushed at the 

% point when timing ball TB overlaps upon timing mark TM on which 
j^J part designating arrow icon Pm is depicted. 

111 When it is judged in step $47 that tne direction key 

20 corresponding to the direction that part designating arrow 
icon Pm is painting has been pushed, progression control 
program 341 carries out an OK judgment for the part 
designation during the processing of step S48. On the other 
hand, when it is judged in step 347 that the direction key 
25 corresponding to tne direction that part designating arrow 
icon Pm is pointing has not been pushed, progression control 
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program 341 carries our a parr designation NG ]udgmenr during 
the processing of step S49. 

Next, based on the symbol (in this case, star mark Sm) 
upon timing mar* tm, whicn mark control program 347 controls; 
the move of timing ball TB, which call control program 348 
controls; and the operating signal of the desired button upon 
the controller, which controller management program 343 
manages, during the processing of step S50, progression 
control program 341 carries out judgment of whether or not 
the desired button has been pushed just for the set duration 
at the point when timing ball TB overlaps upon riming mark 
TM on which part star mark Sm is depicted. 

when it is judged m step 550 that the desired button 
nas been pushed just for the set duration, progression control 
program 341 carries out a sostenuto designation OK judgment 
during rhe processing of step S51. On the other hand, when 
it is judged in step S50 that the desired button has not been 
pushed just for the set duration, progression control program 
341 carries out a sostenuto designation NG judgment during 
the processing of step S52- 

Process F2.ow of Display, Sound Output, Game Point 
Calculat ions and the like in Response to Judgment Results 

Process Flow in Response to OK/NG Judgment Resul t s of 
Timing on the Beat 

In FIG, 17, an example of the process flow of the display 
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and sound output according to tne OK/NG judgment result of 
the timing on the beat from tne processing carried out in step 
S6 of the flowchart of FIG. 14 is shown. 

In FIG. 17, during the processing of step 61, progression 
control program 341 judges if the result of the OK/NG judgment 
for the timing on the beat, which is acquired by the processing 
of stops S41, S42 and S43 of FIG - 16 f is OK or NG . 

When it is judged as OK m step SGI, during the processing 
of step S62, progression control program 341 displays circle 
mark Gm upon timing mark TM corresponding to each beat while 
working together with image control program 344. 

Next, progression control program 341 raises the game 
points during the processing of step S66; moreover, carries 
out the display and sound output of "Good!" or "Bravo!" or 
the luce for every measure auring the processing of step S68, 

On Che other hand, when it is judged as NG in step S6l, 
during the processing of step S63, progression control 
program 341 carries out judgment of whether tne timing of the 
pushing manipulation of the button is slower or faster than 
the timing on the beat. 

When it is judged as Joeing slow m step S63 r during the 
processing of step S64, progression control program 341 
depicts an afterimage of timing ball TB that has passed 
(traversed) timing mark TM as well as depicts x mark Bm upon 
that timing mark TM. 

On the contrary, when it is judged as being fast m step 
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363, during the processing of step S65, progression control 
program 341 depicts X nark Bm upon a timing mark as well as 
depicts an afterimage or riming ball TB just nefore that 
timing mark TM. 

After these steps S64 and S65, progression control 
program 341 lowers the game points daring the processing of 
step S67; moreover, carries oat the display and sound output 
of "Bad!" or a skull mark or the like for each measure during 
the processing of step S69. 

After the processing of steps S68 and S69, during step 
S10 r progression control program 341 calculates the game 
points m response to the judgment results of the timing on 
the bear, proceeding to the next process. 

Process Flow of Display and Sound Output and the like in 
Response to Pressure-Sensed Value Judgment Results 

In FIG. 18, an example of the process flow of the display 
and sound output in response to the pres sure- sensed value 
OK/NG judgment result from the processing carried out in step 
S6 of the flowchart of FIG - 14 is shown. 

In FIG. 19, based on the pressure-sensed value according 
to the pushing pressure on the button, which controller 
management program 343 manages, progression control program 
341, during the processing of step 81, controls the display 
level of pressure-sensing meter 252 as well as carries out 
the setting of the tfolame corresponding to that same 
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pressure-sensed value. 

Next, during the processing of step S82 r progression 
control program 341 judges whether the pressure-sensed value 
OK/NG judgment result, which is acquired by the processing 
of steps S44, S45 and S46 of PIG, 16, is OK or NG . 

When it is judged as OK in step S82, progression control 
program 341 raises the game points during the processing of 
step S83; moreover, carries out the display and sound output 
of "Good!" or "Bravo!" or the like for each measure during 
the processing of step 585, 

On the other hand, when it is judged as NG in step S82, 
progression control program 341 lowers the game points during 
the processing of step 584; moreover, carries out the display 
and sound output of M Bad!" or depicts a skull mark or the like 
for each measure during the processing of step S86. 

After the processing of steps S85 and 586, progression 
control program 341, during step S87, calculates the game 
points according to the pressure-sensed value judgment result, 
proceeding to tne next process. 

It is noted that the case of the process concerning this 
pressure-sensed value judgment result is also similar to the 
case of the OK/NG judgment for the timing on tne beat, 
wherewitn circle mark Gm may be displayed upon each timing 
mark TM in response to the OK/NG judgment result for the 
pressure-sensed value, or contrarily X marJc 3m may £>e 
di splayed . 
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Process Flow of Display and 5ound Output and the like in 
Response to Parr Designation Judgment Results 

In FIG. 19, an example of the process flow of the display 
and sound output according co the part designation judgment 
5 result from the processing carried out m step 56 of the 
flowchart of FIG . 14 is shown. 

In FIG . 19 r during the processing of step 91 , progression 
control program 341 ]udges if the result of the part 
designation judgment, which is acquired by the processing of 
io steps S47 r S48 and S49 of FIG . 16, is OK or NG. 

When it is judged as OK m step 291, during the processing 
of step 592, progression control program 341 carries out 
performing of a part, musical performer or instrument 
corresponding to the direction designated by part designating 
' 1 5 arrow icon Pm worKing together with image control program 344 
and sound control program 345. 

Next, progression control program 341 raises the game 
points during the processing of step S94; moreover, carries 
out the display and sound output of '"Good!" or *Bravol" or 
20 the like for each measure during the processing of step 296. 

On the other hand, when it is nudged as NG in step S91, 
during tne processing or step S93, progression control 
program 341 does not carry out performing of a part, musical 
performer or instrument corresponding to the direction 
25 indicated by part designating arrow icon Pm . 

Next, progression control program 341 lowers the game 
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points during the processing of step S35; moreover, carries 
out the display and sound output of "Bad!" or depicts a skull 
mark or the like for each measure during the processing of 
step S97. 

After steps S96 and S97, progression control program 
341, during step S98, calculates the game points in response 
to the part designation ] adgment result, proceeding to the 
next proces s . 

It is notea that the case of the process concerning this 
part designation judgment result is also similar to tne case 
of the OK/NG judgment for the timing on the beat, wherewith 
circle marK 6m may be displayed upon each timing mark TM m 
response to the OK/NG judgment result for the part designation, 
or contrarily X mark Bm may Joe displayed. 

Process Flow of Display and Sound Output and the like in 
Response to Sostenuto Judgment Results 

In FIG - 20, an example of the process flow of the display 
and sound output in response to the sostenuto OK/NG judgment 
result from the processing carried out in step S6 of the 
flowchart of FIG - 14 is shown. 

In FIG- 20, during tne processing of step S101, 
progression control program 341 judges if the result of the 
sostenuto judgment, which is acquired by the processing of 
steps S50, S51 and S52 of FIG. 16, is OK or NG- 

When it is judged as OK in step S101, during the 
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processing of step S102, progression control program 341 
working together with controller management program 343 and 
image control program 344 displays timing sail TB orbiting 
above timing mark tm whicn has depicted star mark Sm until 
s the desired button upon the controller is manipulated to be 

ON again- 
Next,, progression control program 341 raises the game 
points during the processing of step S103; moreover, carries 
oat the display and sound output of "Good!" or "Bravo I" or 
10 the like for each measure during the processing of step 3105. 

On the other hand, when it is judged as NG in step S101 r 
f»§ progression control program 341 lowers the game points during 

M : the processing of step S104; moreover, carries out the display 

"'4 

m and sound output of "Bad'" or depicts a skull mark or the like 
■His for each measure during the processing of step S106. 

After steps S105 and 5106, progression control program 

a 

i|l 341, during step S107, calculates the game points in response 

to the sostenuto judgment result, proceeding to the next 
W process . 

20 It is noted that the case of the process concerning this sostenuto judgment 

result is also similar to the case of the OK/NG judgment for the timing on the beat, 
wherewith circle mark Gm may be displayed upon each timing mark TM in 
response to the OK/NG judgment result for sostenuto. or contrarily X mark Bin 
may be displayed. 

25 
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TV Game System for Realizing Sound Game 

Next, the schematic structure of TV game system 1 as 
an embodiment capable of realizing such aforementioned sound 
game is illustrated in FIG. 21. 
5 As can be understood from PIG. 21, TV game system 1 is 

an entertainment system comprised of TV game machine 2 as a 
program execution apparatus, controller 20 and television 
monitor 10. TV game machine 2 not only executes the 
aforementioned sound game, but also has the function of 
10 carrying out playback of movies and the like. Controller 20 
is connected to tne TV game machine 2 and comprises various 
buttons and direction keys and the like, which are operation 
terminals that are manipulated by the user (player) . 
Television monitor 10 displays game content and movies and 
15 the like as well as outputs sound. 

Brief Description of TV Game 

TV game machine 2 comprises memory card slots 8A, 8B; 
controller ports 7A, 7B; disk tray 3; buttons 4 and 9; 
zo Institute of Electrical and Electronics Engineers (IEEE) 1394 
connector 6; and Universal Serial Bus (USB) connector 5 and 
the like- Memory card slots 8A, 8B are configured as such 
so that memory card 26 is freely at tachable / removable „ 
Controller ports 7A, IB are configured as such so that 
25 connector 12 of cable 13, which is connected to tne game 
controller 20, is freely attachable/removable. Disk tray 3 
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is configured as such so rhat optical disks, for example, 
DVD-ROMs and CD-ROMs , may be loaded. Button 9 is an 
OPEN /CLOSE button for opening/closing disk tray 3. Button 
4 is an ON / STANDBY /RESET button for turning the power on or 
into standby, or resetting a game. Furthermore, it has been 
omitted from the figures, however, a power supply switch, 
audio/visual output terminal (muiti AV output terminal), PC 
card slot, opt ical /digital output terminal, AC power input 
terminal and the like are provided on tne back panel of the 
TV game machine 2 . 

TV game machine 2 executes games in accordance with 
commands from a player via a game program and controller 20. 
The game program may be either one that is read from various 
recording media for example, optical disks such as the CD-ROMs 
ana DVD-ROMs and semiconductor memory, or one that is 
downloaded via various transmission media such as phone lines, 
LANs, CATV lines and communications satellite lines. It is 
noted that game execution mainly indicates proceeding with 
a game by controlling sound (game scunct) of an audio apparatus 
or images (game images) within a game screen, which is 
displayed upon screen 11 of television monitor 10 connected 
to TV game machine 2, according to commands by the player via 
controller 20- In the case of TV game machine 2 shown in FIG, 
21, two controllers 20 may be connected to controller ports 
7A and 7B, and by utilizing these two controllers, two players 
may carry out various games. For example, various game data 
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generated by execution of tne TV game are scored (saved) on 
memory card 26, which is loaded into memory card slot 8A or 
83; therefore, during subsequent games, continuation and the 
like of the game may be carried oat by utilizing those saved 
game data. 

Moreover, TV game machine 2 can not only execute a TV 
game based on the game program, but can also play back, (decode) , 
for example, audio data, which is recorded on a CD, or a video 
such as a movie or audio data, which is recorded on a DVD. 
Furthermore, it is possible for TV game machine 2 to operate 
based on various application programs other than games. It 
is noted that a ariver program for carrying out play back of 
DVDs is saved on memory card 26, for example. Accordingly, 
TV game machine 2 reads the driver program that is for carrying 
out play bacK of DVDs from memory card 26, whicn is loaded 
into memory card slot 8A or 8B, so as to carry out playing 
back of a DVD based on that driver program, 

Brief Description of Controller 

The controller 20 comprises left grip 35, right grip 
36, left operation pare 21 and right operation part 22, left 
analog operation part 31 and right analog operation part 32, 
first left pushbutton 23L, second left pushbutton not shown 
m the figures, first right pushbutton 23R and second right 
pushbutton not shown in the figures and the like. Left grip 
35 is a portion for the player to grip so as to hold it with 
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the palm of the left hand. Right: grip 36 is a portion for 
the player to grip so as to hold it with the palm of the right 
hand. Left operation part 21 and right operation part 22 are 
portions for the player to respectively operate with the left 
and right thumbs in the state where those grips 35, 36 are 
gripped with the left and right hands. Left analog operation 
part 31 ana right analog operation pair 32 are portions for 
the player to respectively analog operate (operate the 
joystick) with the left and right thumbs in the state where 
grips 35, 36 are gripped with the left and right hands. First 
left pushbutton 23L and the second left pushbutton not shown 
in the figure that are positioned there below are buttons for 
"Che player to respectively operate by depressing with the left 
index finger and middle finger, for example. First right 
pushbutton 23R and the second right pushbutton not shown in 
the figures that are positioned there below are buttons for 
the player to respectively operate by depressing with the 
right index finger and middle finger, for example. 

"Up", "down", "left", "right" direction keys, which the 
player operates when, for example, moving up r down, left or 
right upon a screen a game character, and when carrying out 
instructions for parrs, performer and musical instruments 
(part designations) in the sound garn^, are provided on left 
operation part 21. It is noted that the "up", "down", "left", 
"right'-' direction Keys are not only directional commands of 
upwards, downwards, leftwards or rightwards, out may joe 
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utilized for directional commands in diagonal directions. 
For example, by depressing the >% up" direction Key and "right" 
direction Key simultaneously, the player may give a 
directional command of an upper-right diagonal direction to 
TV game machine 2. It is the same with the other keys, 
wherewith, for example, by depressing the "down" direction 
key and "left" direction Jcey simultaneously, the player may 
give a directional command of a lower-left diagonal direction 
to tv game macnine 2. 

Right operation part 22 is manipulaied to be ON during 
riming of the beat, changing the volume or sostenuto and while 
setting and executing the game character functions; moreover, 
has four command buttons ("A", rt D", "x" r "O" buttons 
respectively provided with A, □, x, O shaped stamps), 
whlcn also function as buttons capable of detecting the 
pushing pressure. £ach command button is respectively 
assigned different functions by the game program. For 
example, the "A" outton is assigned a function for specifying 
menu display; the v x" button , for example, is assigned a 
function for specifying tne deletion and such of selected 
items; the button, for example, is assigned a function 

for specifying the determination and such of selected items; 
and the "O" DUlton, for example, is assigned a function for 
specifying the display/not display of the table of contents 
and such. It is noted that the functions assigned to each 
of these buttons and *eys are an example, wherein various 
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functions may be assigned by the game program* 

Left analog operation part 31 and right analog operation 
part 32 are held in an upright state (non-diagonal state) 
position ( standard position) daring non-inclined operations . 
When left analog operation part 31 and right analog operation 
part 32 are manipulated at an inclination, controller 20 
detects coordinate values of the XY coordinates according to 
the amount of slant and direction in contrast to the standard 
position, wherein those coordinate values are transmitted to 
TV game machine 2 as operation output. 

Furthermore, controller 20 comprises moae selection 
switch 33, light display 34, select button 25, start button 

24 and the like. Mode selection switch 33 is a switch for 
performing tne selection of the operation mode, which makes 
the functions of left and right operation parts 21, 22 and 
left and right analog operation parts 31, 32 operate (analog 
operation mode) or stop (digital operation mode). Light 
display 34 comprises a light emitting diode (LED) for the user 
to recognize the selected operation mode by displaying light. 
Start button 24 is a button for the player to command execution 
start, restart, pause and the like of a game- Select button 

25 is a button for the player to command the displaying and 
saan of the menu display or operation panel upon monitor 
screen 11. it is noted that in the case where the analog 
operation mode is selected by mode selection switch 33, light 
display 34 is controllea to turn ON, resulting m left and 
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right analog operation parts 31, 32 to be in an operating state. 
On the other hand, in the case where the digital operation 
mode is selected, light display 34 is controlled to tarn OFF, 
resulting in left and right analog operation parts 31, 32 to 
be m a non-operating state. 

When various buttons or operation parts upon controller 
20 are manipulated, controller 20 transmits to TV game machine 
2 via cable 3 operation signals according to those operations. 

Moreover, controller 20 also comprises a vibration 
generating mechanism within the left and right grips 35, 36. 
The vibration generating mechanism comprises a weight 
eccentxic from the rotation axis of a motor, for example, and 
the motor rotates that weignt so as to vibrate controller 20. 
This vibration generating mechanism operates in conformity 
with commands from TV game machine 2, By causing the 
vibration generating mechanism to operate, controller 20 
conveys the vibration to the player's hand. 

Internal Configuration of TV Game Machine 

Next, an outline of the internal circuitry of TV game machine 
2 of this embodiment will be described using FIG. 22. 

TV game machine 2 comprises mam CPU 100, graphic 
processing unit (GPU) 110, IO processor (IOP) 120, optical 
disJc player 130, main memory 160, MASK-ROM ISO, and sound 
processing unit (SPU) 140 and the like. Mam CPU 100 performs 
signal processing and the control of internal main structural 
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elements based on various application programs such as game 
programs for realizing the aforementioned sound game. GPU 
110 performs image processing. 10? 120 performs interfacing 
between the exterior and interior of the apparatus ana 
processing for maintaining downward compatibility. Main 
memory 160 has the function of a buffer tnat temporarily 
nouses data read from optical disks or the work area of the 
main CPU 100. MASK-ROM 150 houses, for example, an operating 
system program that main CPU 100 and IOP 120 mainly execute, 
SPU 140 performs acoustic signal processing. Optical disk 
player 130 performs playback of optical disks such as the DVDs 
and CDs, which are recorded with application programs or 
multi-media data- furthermore, optical disk player 130 is 
comprised of a spindle motor, optical pickup, RF amp 131 and 
sliding mechanism and the like- The spindle motor rotates 
optical disks such as DVDs and CDs - The optical pickup reads 
the signals that are recorded on the optical disks. The 
sliding mechanism moves the optical pickup along the disk 
radius- RF amp 131 amplifies the output signal from the 
optical pickup. 

Moreover, TV game machine 2 also has CD/DVD digital 
signal processor 170 (nereafter notated as DS? 170), driver 
180, mechanical controller 190 and card-type connector 200 
(hereafter notated as PC card slot 200) - Sy digitizing output 
signals from RF amp 131 of optical disk player 130, and 
applying, for example, error correction processing { c 1 RC 
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processing) or expand/encode processing and the liJce, DSP 170 
plays back signals thai are recorded on the optical disks. 
Driver 180 and mechanical controller 190 perform rotation 
control of the spindle motor of optical disk player 130; focus 
and tracking control of the optical pickup; and loading 
control of the disK tray and the like. PC card slot 200 is 
an interface device for connecting, for example, a 
communication card and external hard disk drive and the like. 

Each of these parts is respectively connected, to the 
other mainly via bus lines 202, 203 and the like. It is noted 
that main CPU ICO and GPU 110 are connected by a dedicated 
bus; furthermore, mam CPU 100 and IOP 120 are connected £>y 
the SBUS* IOP 120, DSP 170, MASK-ROM 150, SPU 140 and PC card 
slot 200 are connected by the SBUS. 

By executing an operating system program for the main 
CPU that is stored on MASK-ROM 150, mam CPU 100 controls all 
operations of TV game machine 2. Mam CPU 100 controls 
operations of the game and the like of TV machine 2 by executing 
various application programs and the like including game 
programs for sound games, which are read from optical disks, 
for example, CD-ROMs and DVD-ROMs and the like, so as to be 
loaded into mam memory 160, or downloaded via communication 
networks - 

By executing an operating system program for the IOP, 
which is stored on MASK-ROM 150, IOP 120 controls the 
input /output of data from memory card 26, which stores signals 
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and game settings and the like from controller 20 according 
to the player's operations, and input /output of data of USB 
connector 5, IEEE 1394 connector 6, PC card slot 200 and the 
like, as well as performs data protocol conversion and the 
like . 

GPU 110 has the function of a geometry transfer engine 
that performs processing of coordinate transformation and the 
like, and the function of a rendering processor; and performs 
drawing according to the drawing command from main CPU 100 
so as to house the drawn images in a frame buffer not shown 
in the figures. More specifically, for example, in the case 
where various application programs, which are recorded on 
optical disks, are utilized for so-called 3 -dimensiona 1 (3D) 
graphics such as TV games, the GPU 110 performs coordinate 
calculations and the like by geometric operation processing 
of polygons for configuring 3-dimensional objects; moreover, 
calculations, namely perspective transformation 
(calculations of coordinates and such of the vertexes of each 
polygon comprising a 3-dimensional object when it is 
projected upon a virtual camera screen) are performed by 
rendering processing for generating an image, which is 
acquired by photographing this 3-dimensional object with a 
virtual camera so as to write the ultimately acquired image 
data upon the frame buffer. Then, GPU 110 outputs Video 
Signals corresponding to this formed image. 

S?U 140 comprises an adaptive differential pulse code 
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modulation (ADPCM) decode function, audio signal playback 
function, signal modulation function and the like. The ADPCM 
decode function is a function for decoding acoustic data that 
has been adaptive prediction encoded. The audio signal 
playback function is a function for playing back and 
outputting audio signals such as sound effects by reading 
waveform data, which is stored in an internal or external 
sound ouffer (not shown in the figures) of the SPU 140. The 
signal modulation function is a function for generating 
various sound waveform dara by modulating waveform data, 
wnicn is stored in the sound buffer. More specifically, the 
SPU 140, based on commands from main CPU 100, generates audio 
signals such as musical tones and sound effects and the like 
from waveform data, which is stored in the sound buffer, and 
also operates as a so-called sampling sound source. 

When TV game machine 2 having the configuration as the 
above, for example, is energized, an operating system program 
for the mam CPU and an operating system program for the IOP 
are respectively read from MASK-ROM 150, wherewith the 
corresponding operating system program for main CPU 100 and 
IOP 120 are executed. Accordingly, main CPU 100 centrally 
controls each part of TV game machine 2* IOP 120 controls 
tne input/output of signals with controller 20, memory card 
26 and the like. wnen tne operating system program is 
executed, after main CPU 100 performs initialization 
processing such as checking of operations and the like, a game 
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program that is recorded on an optical disk is read by 
controlling optical disk player 130, wherewith after loading 
into main memory 160/ that game program is executed. By the 
execution of this game program, main CPU 100 controls GPU 110 
and SPU 140 according to the player's command accepted from 
controller 20 via IOP 120, and controlling the display of 
images and generarion of sound effects and musical tones. It 
is noted that the case where playback of movies and the like 
recorded on optical disks, for example, is the same as with 
TV game machine 2. Namely, main CPU 100 controls GPU 110 and 
SPU 140 according to the player's command accepted from 
controller 20 via IOP 120, and controlling the display of 
movie images played back from optical aisles and the generation 
of sound effects and musical tones and the like. 



Means for Detecting P ushing Pressure during Button ON 
Manipulation 

The "A" r -x", "O" buttons of right operation part 

22 of controller 20 are respectively configured as 
pressure-sensitive buttons capable of detecting pushing 
pressure . 

PIG. 23 through TIG. 25 show cross sections cut along 
a certain line running in the pushing pressure direction on 
pressure-sensitive bution 60. It is noted that FIG. 23 shows 
pressure-sensitive button 60 in a non-pushed state; FIG. 24 
shows pressure-sensitive button 60 in a partly pushed state; 
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and FIG. 25 shows pressure-sensitive button 60 in a fully 
pushed state. 

As Shown m FIG. 23, pressure-sensmve button 60 has 
substrate 55; pad 53, which is comprised of flexible material 
sucn as rubber or a spring or the like; pressure member 52 r 
which is comprised of flexible material such as rubber or a 
spring or the like; and swirch cover 50, which is made of hard 
plastic such as ABS resin and the like or rubber or the like. 
Pressure-sensi:ive resistor 54, the resistance of which 
changes based on the contact area that makes contact with the 
object, is configured upon substrate 55. Pad 53 has rim 53a f 
which is fixated upon substrate 55; middle portion 53b, which 
is configured derached from substrate 55 so as to form a 
predetermined opening between pressure-sensitive resistor 54 
and said middle portion; and wall 51, which int egrat ively 
links rim 53a to middle portion 53b. Pressure member 52 is 
fixed on middle portion 53b within the opening, facing 
pressure sensitive resistor 54. Switch cover 50 is mounted 
upon middle portion 53b o£ pad 53, and is movable along with 
middle portion 53. 

When the player depresses pad 53, wall 51 bends to 
transform in response to that pushing pressure; when the 
pushing pressure reaches a predetermined value or more, 
pressure-sensitive resistor 54 and pressure member 52 make 
contact as shown in FIG. 24 and FIG. 25. 

Surface 52a of pressure member 52 that faces 
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pressure-sensitive resistor 54 has a rough conic shape with 
its central portion projecting out most. Since pressure 
member 52 is also formed of flexible material such as rubber 
and the llice, m response to the pushing pressure applied by 
the operator (player) „ surface 52a of pressure member 52 
transforms by touching and being crushed by pressure 
sensitive-resistor 54 at its central portion as shown in FIG - 
24 and FIG. 25. It is noted that FIG. 2d and FIG. 25 
respectively show the state where only tne central portion 
(tip portion) of pressure member 52 touches pressure 
sensitive-resistor 54 r and the state where the area of contact 
with pressure-sensitive resistor 54 and pressure member 52 
is at tne maximum (the state where the entirety of side 52a 
touches pressure-sensitive resistor 54), 

The resistance value of pressure-sensitive resistor 54 
varies oasea on tne area of contact with such pressure member 
52 (in otner words, in response to Che pushing pressure) ; 
becomes a large resistance value when the contact area is 
small, and as the contact area expands, gradually changes to 
a small resistance value. 

FIG. 26 shows the schematic circuitry o£ 
pressure-sensitive resistor 54, the resistance of which 
changes in response to the pushing pressure (the pushing 
pressure upon ores sure - s ens i t i ve button 60). 

As shown in FIG. 26, pressure-sensitive resistor 54 is 
inserted in series in the power supply line between r he 
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reference voltage (vcc) and grouna. Pressure-sensiiive 
rcsisior 54 may be schematically illustrated by fixed 
resistor R and variable resistor VR. 

Among these, trie portion of the variable resistor VR 
corresponds to the contact portion of pressure member 52, 
wnerewitn its resistance value varies according to the 
contact area of pressure member 52. Namely, when pressure 
member 52 is in contact with pressure-sensitive resistor 54, 
since current flows with this pressure member 52 becoming a 
bridge, the resistance value of that contact portion 
(variable resistor VR) becomes smaller. 

in this embodiment, the portion of the variable Resistor 
VR is connected to the ground side of the power supply line. 
T n e n , the analog signal in response to the pushing pressure 
upon button 60 is taken out from between the portion of fixed 
resistor R and the portion of variable resistor VR of 
pressure-sensitive resistor 54. As a result, the resistance 
value of variable resistor VR becomes smaller as the pushing 
pressure upon button 60 is weakened; concurrently, the 
voltage level of the analog signal taken out from between 
fixed resistor R ana variable resistor VR increases. 

FIG » 27 is a graph showing variances in the pushing 
pressure (horizontal axis) that is applied to 
pressure-sensitive button 60 and the voltage values {vertical 
axis) of tne electric signal corresponding to that pushing 
pressure. As can be understood from PIG. 27, in the state 
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where pushing pressure is nor applied to pressure-ser.smve 
barton 60 cpashing pressure is zero), since pressure member 
52 is not in touch with pressure sensitive-resistor 54, the 
resistance value of pressure-sensitive resistor 54 is its 
maximum value. As a resulr, in this case, the maximum voltage 
(Vmax) is taken out from between fixed resistor R and variable 
resistor VR as the analog signal that indicates the present 
pushing pressure. 

Wnen pusning pressure is applied to pressure-sensitive 
button 60 and that pushing pressure reaches pressure PI, 
pressure member 52 begins to touch pressure-sensitive 
resistor 54, whereby concurrently with the pusning pressure 
being increased from pressure PI, the resistance value cf 
pressure-sensitive resistor 54 gradually diminishes. wnen 
the pushing pressure reaches pressure P2, the contact area 
of pressure-sensitive resistor 54 and pressure member 52 
becomes its maximum size, whereby the resistance value of 
pressure-sensitive resistor 54 becomes its minimum value. 
At this time, the minimum voltage (Vmin) is taken out from 
between fixed resistor R and variable resistor VR as an analog 
signal that indicates the pushing pressure, 

in this manner, the analog signal taken out from between 
fixed resistor R and variable resistor VR i.i transmitted to 
TV game machine 2 from controller 20 of FIG. 21 as the 
pressure-sensed value that indicates the pushing pressure at 
that time; moreover, it is transmitted to CPU 100 from I/O 
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processor (IOP) 120 shown in FIG. 22 so as to carry our judgment 
and the llKe by CPU 100 in response to such aforementioned 
pressure-sensed values. 

As described above, the game program of the sound game, 
by such as the distance among timing marks TM, number of marks , 
display color of each rcark, the sign displayed upon the mark, 
and tne moving of timing ball TB, visually informs the player 
of each element, such as tempo, rhythm, beat, volume, part 
designations and sostenuto, that are necessary for when 
carrying out simulated conducting and producing of music. 
Furthermore, the game program carries out adjustment of tne 
duration for sostenuto by detecting the ON manipulation of 
the button upon the controller' by the player, and detects the 
pushing pressure on tnat button so as to carry out volume 
adjustment according to that pressure- sensed value; moreover, 
by detecting manipulation of the direction keys upon the 
controller by the player, allowing the player to carry out 
simulated conducting and producing of music. 

It is noted that the above description of the embodiment 
is an example of the present invention. As a result, the 
present invention is not limited to the embodiment described 
above, and needless to say, various modifications thereto 
according to design of choice and such can be made witnout 
deviating from the technical spirit and scope of the present 
invent ion - 

Namely, the present invention is not on© to be limited 
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to sound games that are played id accordance with such musical 
performance as described aoove, but is also applicable to 
.situations without music. 

The present invention is also applicable to games that 
display timing marks in accordance with, for example, 
arrangements of dance steps so as to move a timing ball in 
accordance with actions to the steps. 

Each of the elements needed when carrying out simulated 
conducting or producing of music is not limited to the six 
types of tempo, rhythm, beat, volume, part designations, and 
sos t enut o . 

The color and symbol and the like of each timing mark 
are not limited to the above example, wherewith by changing 
the display parameters for when displaying the timing marks 
upon the monitor screen, for example, tne level of volume 
(weak/strong performance} may be expressed by the brigntness 
of each timing mark instead of the color. 

The timing marks and timing ball are not limited to the 
virtual objects displayed upon the television monitor screen, 
but the present invention may also be applicable to the 
structure such that a substantive object that corresponds to 
the timing ball moves among substantive objects tnat 
correspond to the timing marks, for example. 

The snaps formed by each timing mark is not limited to 
such equilateral triangle as in FIG. 1 and the li>ce or square 
as in FIG- 3, tout may be, for example, an isosceles triangle 
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or trapezoid; the moving route of the riming ball may be nor 
only a route of moving to eacft riming mark in turn, but may 
be a route where predetermined timing marks are skipped; and 
the moving speed of the timing Gall is not limited to a fixed 
speed . 

The set of timing marks and timing ball displayed upon 
the monitor screen may be not only one set but also may be 
a plurality of sets. 

In addition, the part designations may be carried our 
by, for example, left analog operation part 31 other than tne 
direction keys upon the controller. 

While preferrea embodiments of the present invention 
have been described using specific terms, such description 
is for illustrative purposes, and ir is to be understood that 
changes ana variations may be made without departing f rem the 
spirit or scope of the following claims. 

As described above, the present invention expresses, 
by the number of predetermined objects, a structure that 
repeats periodically changing information (for example, beat 
of the music) , and expresses the progressing speed of the 
information (for example, tempo of the music) by the moving 
of a moving object and the distance among the predetermined 
objects. As a result, the player may easily grasp the timing 
of the predetermined information and easily determine the 
cycle of the information . Namely, the player may, for example , 
easily grasp the tempo of the music m the sound game, and 
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furthermore may easily determine the bear of the piece. 
Accordingly, it becomes possible for the player to personally 
carry out a musical production, for example, as an orchestral 
conductor . 
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